For low-level computer vision and image processing ML tasks, training on large datasets is critical for generalization. However, the standard practice of relying on real-world images primarily from the Internet comes with image quality, scalability, and privacy issues, especially in commercial contexts. To address this, we have developed a procedural synthetic data generation pipeline and dataset tailored to low-level vision tasks. Our Unreal engine-based synthetic data pipeline populates large scenes algorithmically with a combination of random 3D objects, materials, and geometric transformations. Then, we calibrate the camera noise profiles to synthesize the noisy images. From this pipeline, we generated a fully synthetic image denoising dataset (FSID) which consists of 175,000 noisy/clean image pairs. We then trained and validated a CNN-based denoising model, and demonstrated that the model trained on this synthetic data alone can achieve competitive denoising results when evaluated on real-world noisy images captured with smartphone cameras.
translated by 谷歌翻译
Diffusion models have emerged as a powerful tool for point cloud generation. A key component that drives the impressive performance for generating high-quality samples from noise is iteratively denoise for thousands of steps. While beneficial, the complexity of learning steps has limited its applications to many 3D real-world. To address this limitation, we propose Point Straight Flow (PSF), a model that exhibits impressive performance using one step. Our idea is based on the reformulation of the standard diffusion model, which optimizes the curvy learning trajectory into a straight path. Further, we develop a distillation strategy to shorten the straight path into one step without a performance loss, enabling applications to 3D real-world with latency constraints. We perform evaluations on multiple 3D tasks and find that our PSF performs comparably to the standard diffusion model, outperforming other efficient 3D point cloud generation methods. On real-world applications such as point cloud completion and training-free text-guided generation in a low-latency setup, PSF performs favorably.
translated by 谷歌翻译
现有的数据依赖性哈希方法使用具有数百万个参数的大型骨干网络，并且计算复杂。现有的知识蒸馏方法使用深（教师）模型的逻辑和其他功能，并将其作为紧凑型（学生）模型的知识，这要求教师的网络在上下文中与上下文中的学生模型平行进行微调。在目标环境中培训老师需要更多的时间和计算资源。在本文中，我们提出了不知道知识蒸馏的上下文，该蒸馏使用教师模型的知识而不在目标环境上进行微调。我们还提出了一种新的高效学生模型架构，用于知识蒸馏。提出的方法遵循两步过程。第一步涉及在不知道教师模型的不知道知识蒸馏的情况下预先培训学生模型。第二步涉及在图像检索的上下文上微调学生模型。为了显示拟议方法的功效，我们比较了检索结果。参数和否。在不同检索框架下，学生模型的运营与教师模型的运作，包括Deep Cauchy Hashing（DCH）和中央相似性量化（CSQ）。实验结果证实，所提出的方法在检索结果与效率之间提供了有希望的权衡。本文中使用的代码通过\ url {https://github.com/satoru2001/cukdfir}公开发布。
translated by 谷歌翻译
视频修复旨在从多个低质量框架中恢复多个高质量的帧。现有的视频修复方法通常属于两种极端情况，即它们并行恢复所有帧，或者以复发方式恢复视频框架，这将导致不同的优点和缺点。通常，前者具有时间信息融合的优势。但是，它遭受了较大的模型尺寸和密集的内存消耗；后者的模型大小相对较小，因为它在跨帧中共享参数。但是，它缺乏远程依赖建模能力和并行性。在本文中，我们试图通过提出经常性视频恢复变压器（即RVRT）来整合两种情况的优势。 RVRT在全球经常性的框架内并行处理本地相邻框架，该框架可以在模型大小，有效性和效率之间实现良好的权衡。具体而言，RVRT将视频分为多个剪辑，并使用先前推断的剪辑功能来估计后续剪辑功能。在每个剪辑中，通过隐式特征聚合共同更新不同的帧功能。在不同的剪辑中，引导的变形注意力是为剪辑对齐对齐的，该剪辑对齐可预测整个推断的夹子中的多个相关位置，并通过注意机制汇总其特征。关于视频超分辨率，DeBlurring和DeNoising的广泛实验表明，所提出的RVRT在具有平衡模型大小，测试内存和运行时的基准数据集上实现了最先进的性能。
translated by 谷歌翻译
视频修复（例如，视频超分辨率）旨在从低品质框架中恢复高质量的帧。与单图像恢复不同，视频修复通常需要从多个相邻但通常未对准视频帧的时间信息。现有的深度方法通常通过利用滑动窗口策略或经常性体系结构来解决此问题，该策略要么受逐帧恢复的限制，要么缺乏远程建模能力。在本文中，我们提出了一个带有平行框架预测和远程时间依赖性建模能力的视频恢复变压器（VRT）。更具体地说，VRT由多个量表组成，每个量表由两种模块组成：时间相互注意（TMSA）和平行翘曲。 TMSA将视频分为小剪辑，将相互关注用于关节运动估计，特征对齐和特征融合，而自我注意力则用于特征提取。为了启用交叉交互，视频序列对其他每一层都发生了变化。此外，通过并行功能翘曲，并行翘曲用于进一步从相邻帧中融合信息。有关五项任务的实验结果，包括视频超分辨率，视频脱张，视频denoising，视频框架插值和时空视频超级分辨率，证明VRT优于大幅度的最先进方法（$ \ textbf） {最高2.16db} $）在十四个基准数据集上。
translated by 谷歌翻译
针对边缘设备的实用眼睛认证（EA）系统需要对呈现攻击进行身份验证并强大，同时剩余计算和延迟效率。然而，现有的基于眼框架A）独立地执行认证和呈现攻击检测（PAD），B）涉及提取虹膜区域的显着预处理步骤。在这里，我们使用围绕图像介绍EA和垫的联合框架。虽然深度多任务学习（MTL）网络可以执行任务，但由于EA和焊盘的训练数据集是不相交的，因此MTL遭受遗忘效果。为了克服这一点，我们提出了用垫（眼部）的眼睛认证，一种基于蒸馏的方法，该方法为EA和垫训练了一个网络，同时降低了遗忘的效果。为了进一步提高EA性能，我们介绍了一种名为Eyepad ++的新方法，包括在EA和焊盘数据上训练MTL网络，同时通过额外的蒸馏步骤蒸馏眼网网络的“通用性”。我们所提出的方法优于垫中的SOTA，并在眼睛验证中获得近的SOTA性能，而无需任何预处理。我们还展示了眼部和眼部++在用户到用户验证中的疗效，跨网络骨干网和图像质量。
translated by 谷歌翻译
We demonstrate how efficient autonomous drone swarms can be in detecting and tracking occluded targets in densely forested areas, such as lost people during search and rescue missions. Exploration and optimization of local viewing conditions, such as occlusion density and target view obliqueness, provide much faster and much more reliable results than previous, blind sampling strategies that are based on pre-defined waypoints. An adapted real-time particle swarm optimization and a new objective function are presented that are able to deal with dynamic and highly random through-foliage conditions. Synthetic aperture sensing is our fundamental sampling principle, and drone swarms are employed to approximate the optical signals of extremely wide and adaptable airborne lenses.
translated by 谷歌翻译
In this paper, we propose SceNDD: a scenario-based naturalistic driving dataset that is built upon data collected from an instrumented vehicle in downtown Indianapolis. The data collection was completed in 68 driving sessions with different drivers, where each session lasted about 20--40 minutes. The main goal of creating this dataset is to provide the research community with real driving scenarios that have diverse trajectories and driving behaviors. The dataset contains ego-vehicle's waypoints, velocity, yaw angle, as well as non-ego actor's waypoints, velocity, yaw angle, entry-time, and exit-time. Certain flexibility is provided to users so that actors, sensors, lanes, roads, and obstacles can be added to the existing scenarios. We used a Joint Probabilistic Data Association (JPDA) tracker to detect non-ego vehicles on the road. We present some preliminary results of the proposed dataset and a few applications associated with it. The complete dataset is expected to be released by early 2023.
translated by 谷歌翻译
A hallmark of human intelligence is the ability to learn new concepts purely from language. Several recent approaches have explored training machine learning models via natural language supervision. However, these approaches fall short in leveraging linguistic quantifiers (such as 'always' or 'rarely') and mimicking humans in compositionally learning complex tasks. Here, we present LaSQuE, a method that can learn zero-shot classifiers from language explanations by using three new strategies - (1) modeling the semantics of linguistic quantifiers in explanations (including exploiting ordinal strength relationships, such as 'always' > 'likely'), (2) aggregating information from multiple explanations using an attention-based mechanism, and (3) model training via curriculum learning. With these strategies, LaSQuE outperforms prior work, showing an absolute gain of up to 7% in generalizing to unseen real-world classification tasks.
translated by 谷歌翻译
Graph neural networks have achieved significant success in representation learning. However, the performance gains come at a cost; acquiring comprehensive labeled data for training can be prohibitively expensive. Active learning mitigates this issue by searching the unexplored data space and prioritizing the selection of data to maximize model's performance gain. In this paper, we propose a novel method SMARTQUERY, a framework to learn a graph neural network with very few labeled nodes using a hybrid uncertainty reduction function. This is achieved using two key steps: (a) design a multi-stage active graph learning framework by exploiting diverse explicit graph information and (b) introduce label propagation to efficiently exploit known labels to assess the implicit embedding information. Using a comprehensive set of experiments on three network datasets, we demonstrate the competitive performance of our method against state-of-the-arts on very few labeled data (up to 5 labeled nodes per class).
translated by 谷歌翻译